Add native LLM core foundation#24712
Merged
kitlangton merged 199 commits intodevfrom May 8, 2026
Merged
Conversation
This was referenced Apr 29, 2026
- Structurally match recorded requests by canonical JSON so non-deterministic field ordering doesn't break replay. - Pluggable header allow-list and body redaction hook on the record/replay layer, so adapters with non-default auth (Anthropic, Bedrock) can plug in without touching this file. - Move the cassette-name dedupe set inside recordedTests() so two describe files using different prefixes can run in parallel. - Replace inline SSE template literals and per-file HTTP layers with shared test/lib helpers (sseEvents, fixedResponse, dynamicResponse, truncatedStream). - Tighten recorded-test assertions to exact text and usage so adapter parser regressions surface immediately instead of passing fuzzy length>0 checks. - Add cancellation and mid-stream transport-error tests for the OpenAI Chat adapter. - Add cross-phase patch tests that verify each phase sees an updated PatchContext and that same-order patches sort deterministically by id.
- shared sse helper now expects Effectful decodeChunk and process callbacks, so adapter parsers can be Effect.gen and yield typed ProviderChunkError instead of throwing across the sync mapAccum boundary. - parseJson returns Effect<unknown, ProviderChunkError> via Effect.try, matching the package style guide on yieldable errors. - OpenAI Chat finalizes accumulated tool inputs eagerly when finish_reason arrives, surfacing JSON parse failures at the boundary instead of at halt. onHalt stays sync and just emits from state. - generate's runFold reducer now mutates the accumulator instead of reallocating the events array on every chunk, dropping O(n^2) growth on long streams.
Gemini rejects integer enums, dangling required fields, untyped arrays, and object keywords on scalar schemas. The sanitizer was previously a divergent copy in OpenCode; this lands it in the package as a tool-schema patch with deterministic tests and selects it for Gemini-protocol or Gemini-named models. Also tightens the Gemini test suite: covers tool-choice none, drops the tool-input-delta assertion that Gemini does not actually emit, and confirms total usage stays undefined when only thoughtsTokenCount arrives.
…integration Updates the AGENTS.md TODO list: - mark Responses, Anthropic, and Gemini adapter coverage as done - mark the Gemini schema sanitizer port as done - add concrete next-step items for OpenCode integration: ModelRef bridge, request bridge, provider-quirk patches, request/stream parity tests, and a flagged rollout against existing session/llm.test.ts cases - add OpenAI-compatible Chat, Bedrock Converse, and Vertex routing as outstanding adapter/dispatch decisions
Every adapter's parse already produces LLMEvents (via the process callback in the shared sse helper), and every raise was Stream.make(event). The Chunk type parameter, the raise field, the RaiseState interface, and the Stream.flatMap raise step in client.stream were all pure overhead. - Adapter contract shrinks from <Draft, Target, Chunk> to <Draft, Target>. - All four adapters drop their raise: (event) => Stream.make(event) line. - client.stream skips the no-op flatMap. - AGENTS.md adapter section reflects the simpler contract.
Per the package style guide, sync if/return functions that need to fail should yield the error directly via Effect.gen rather than ladder Effect.fail / Effect.succeed across every branch. Touches all four adapters' tool-choice lowering. The naming-required validation now reads as 'guard, then return' rather than embedded in a chain of monadic returns. Behavior unchanged.
Locks down the error contract before OpenCode integration: - mid-stream provider errors (Anthropic 'event: error', OpenAI Responses 'type: error') surface as 'provider-error' LLMEvents - HTTP 4xx responses fail with ProviderRequestError before stream parsing begins (the executor contract) Anthropic already had both. Adds: - OpenAI Responses: provider-error fixture, code-fallback fixture, HTTP 400 - OpenAI Chat: HTTP 400 sad path - AGENTS.md TODO refreshed; live recordings of provider errors still pending
Schema-first, Effect-first tool loop:
- 'tool({ description, parameters, success, execute })' constructs a fully
typed Tool. parameters and success are Effect Schemas; execute is typed
against them and returns Effect<Success, ToolFailure>. Handler dependencies
are closed over at construction time so the runtime never sees per-tool
services.
- 'ToolRuntime.run(client, { request, tools, maxSteps?, stopWhen? })' streams
the model, decodes tool-call inputs against parameters, dispatches to the
matching handler, encodes results against success, emits tool-result events,
appends assistant + tool messages, and re-streams. Stops on non-tool-calls
finish, maxSteps, or stopWhen.
- Three recoverable error paths emit tool-error events so the model can
self-correct: unknown tool name, input fails parameters Schema, handler
returns ToolFailure. Defects fail the stream.
- 'ToolFailure' added to the schema and exported as the single forced error
channel for handlers.
- Tool definitions on the LLMRequest are derived via toJsonSchemaDocument so
consumers don't write JSON Schema by hand.
8 deterministic fixture tests cover the loop, errors, maxSteps, stopWhen, and
parallel tool calls in one step.
Three small fixes for stale references after the recent adapter→route rename and `model.protocol` field removal in @opencode-ai/llm: - Update three `@opencode-ai/llm/adapter` imports to `/route`. - Switch `NATIVE_PROTOCOLS` filter on `model.protocol` to `NATIVE_ROUTES` on `model.route`. - Refresh test fixtures: `protocol:` → `route:`, drop `apiKey:` / `adapter:` assertions that reference fields no longer on `ModelRef`, drop the `attachments` expectation that no longer has a source-side handler. - Default `key: "test-key"` on `ProviderTest.info` so prepare() can resolve auth without per-test env setup.
Push URL host knowledge from `Endpoint` (route layer) up to `model.baseURL`
(provider helper layer). The route just composes a path onto whatever
host the model already carries.
Endpoint:
- `Endpoint<Body>` shrinks to `{ path }`. No more `default`, `baseURL`,
or `required` fallback fields.
- `Endpoint.path(value)` replaces the old `Endpoint.baseURL({...})` factory.
- `Endpoint.render()` is now sync — `${model.baseURL}${path}` plus query
params. No fallback chain, no Effect wrapper.
ModelRef:
- `ModelRef.baseURL: Schema.String` (was optional). Every materialized
model carries a host.
- `RouteModelInput.baseURL?: string` stays optional so route defaults
can supply a canonical URL; routes without a default tighten it.
Provider helpers:
- Each protocol exports a `DEFAULT_BASE_URL` constant and bakes it into
`route.defaults.baseURL`. Provider helpers don't need to set baseURL.
- Azure uses a new `AtLeastOne<T>` helper from `auth-options.ts` to
require either `resourceName` or `baseURL` at the type level.
- Bedrock provider computes baseURL from `region` at construction time.
- OpenAI-compatible profiles now have required (not optional) `baseURL`
in their type — all 9 already supplied one.
- The `defaultBaseURL: string | false` knob on protocol endpoint
factories is gone.
Effects:
- Forgetting baseURL is now caught at compile time (TS) or model
construction time (modelWithDefaults runtime check), not request time.
- `Endpoint.render` no longer needs Effect wrapping in the transport
hot path.
Bring the package guide in line with everything that's landed (route
rename, body/event vocabulary, schema split, endpoint simplification,
WebSocket transport, AuthOptions.bearer, dispatched protocol step).
- Update AGENTS.md throughout: payload→body, chunk→event, processChunk→step,
Endpoint.baseURL({...})→Endpoint.path(...), refreshed folder layout,
refreshed Routes / URL Construction / Provider Definitions sections.
- Fold HOUSE_STYLE.md (protocol file shape, rules, review checklist) into
AGENTS.md as a "Protocol File Style" section.
- Delete the four DESIGN.*.md proposals that are fully implemented:
routes-protocol-transport, websocket-transport, http-retry, model-options.
- Delete TOUR.md — it had grown into a 700-line narrative walkthrough that
duplicated AGENTS.md with stale vocabulary. example/tutorial.ts is the
canonical reading path now.
Three small cleanups in the OpenAI Responses protocol:
- Unify `HOSTED_TOOL_NAMES` + `hostedToolInput` into one `HOSTED_TOOLS`
record per tool: `{ name, input: (item) => unknown }`. Adding a new
hosted tool is now a single entry instead of two parallel switches that
must stay in sync.
- Tighten `isHostedToolItem`'s narrowing to include the `type` field, so
callers know they're dealing with a known hosted-tool shape (not just
"has an id"). Drives a cleaner `hostedToolEvents` signature.
- Split the body schema into a shared `OpenAIResponsesCoreFields` record
used by both the HTTP body (adds `stream: true`) and the WebSocket
`response.create` message (adds `type`). Removes the destructure-and-
strip dance at schema definition time. Runtime conversion in
`webSocketMessage` still strips `stream` because OpenAI's WebSocket API
doesn't expect it on the wire.
Plus a tiny fix in bedrock-converse.ts: explicit `Route.model<BedrockConverseModelInput>`
type argument so the mapInput overload selects properly (was inferring
to the narrower `RouteModelInput`).
Capture the assessment of how opencode integrates the AI SDK today and the phased plan to replace it with @opencode-ai/llm behind a feature flag. Sections: - Today's architecture, including the trace of one streamText call and the existing native path's gate conditions - Where the spaghetti actually lives (AI SDK type leakage in 11+ files, scattered provider-specific transforms, the provider/sdk/copilot/* fork, duplicated MessageV2 conversion) - Target architecture (one flag, one decision point at layer construction) - Phased migration: Decouple → Service swap → Native parity → Flag rollout → Delete AI SDK - Suggested execution order, key files, risks and open questions
# ------------------------ >8 ------------------------ # Do not modify or remove the line above. # Everything below it will be ignored. # # Conflicts: # bun.lock
katosun2
pushed a commit
to katosun2/opencode
that referenced
this pull request
May 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
packages/llm, a native Effect-based LLM core with typed request/event schemas, route/protocol/provider composition, tool runtime, structured object generation, and recorded provider golden tests.@opencode-ai/http-recorder, a private workspace test helper for deterministic HTTP/WebSocket cassette recording and replay used by the LLM package.Safety
Testing
cd packages/llm && bun typecheckcd packages/llm && bun run testcd packages/http-recorder && bun typecheckcd packages/http-recorder && bun test